Acquirement of audio-visual feature correspondence from mere experience

Temporal Structure and Complexity Affect Audio-Visual Correspondence Detection

Synchrony between events in different senses has long been considered the critical temporal cue for multisensory integration. Here, using rapid streams of auditory and visual events, we demonstrate how humans can use temporal structure (rather than mere temporal coincidence) to detect multisensory relatedness. We find psychophysically that participants can detect matching auditory and visual st...

Creation and Selection of the Visual Front End Features and the Audio-Visual Feature Fusion for Audio-Visual Speech Recognition

This contribution is about a creation and selection of the visual front end speech features. The use of the visual shape and the appearance-based visual features are described here. These visual features can be used for the visual or for the audiovisual speech recognition. Before they are used, the features have to be normalized and selected in such a way, so that the recognition rate was high ...

Combining Feature- and Correspondence-Based Methods for Visual Object Recognition

We present an object recognition system built on a combination of feature- and correspondence-based pattern recognizers. The feature-based part, called preselection network, is a single-layer feedforward network weighted with the amount of information contributed by each feature to the decision at hand. For processing arbitrary objects, we employ small, regular graphs whose nodes are attributed...

Feature Extraction from Visual Data Feature Extraction from Visual Data

Audio – Visual Feature Extreaction for Phoneme Recognition of Meetings